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ABSTRACT 


The study aimed to develop the scoring framework for the self- 
rating proficiency indicators for Philippine STEAM educators. The 
study further sought to design mathematical framework and program 
for scoring STEAM educators’ proficiency and validate the designed 
scoring program. About 1507 responses from the self-rating STEAM 
proficiency tool were used to undergo the three tier quantitative and 
qualitative validation. Mathematical equations were derived to direct 
the development of the scoring programs using Microsoft Excel and 
Fortran. Results show an agreement between the proficiency profiles 
generated from the Microsoft Excel and Fortran program. Using the 
online survey and the classroom observation rating, proficiencies were 
compared as determined through the Fortran program of the pre- 
determined career stage (distinguished, highly proficient, proficient, 
beginner). Lastly, qualitative validation was performed by comparing 
the generated codes in the interview transcripts and observation notes 
and the attributes in the PPST domains and TPACK dimensions. 
Qualitative validation indicates that the occurrences of the indicators in 
the interview and classroom observation matched with the expected 
attributes per career stage as per the PPST. This indicates that the 
validation of the scoring system developed for the online survey 
generate the STEAM educator proficiency. Further, the scope of the 
scoring framework developed is universal and may be adapted to suit 
any local setting. However, increasing the number of interviews and 
classroom observations to 10% of the sample population of teachers will 
produce a robust scoring program. 


Keywords: STEAM education, TPACK dimension, PPST domain, 
STEAM educators’ proficiency, STEAM proficiency scoring program 


INTRODUCTION 

Education seems to be the basis of every country’s progress and 
groundwork for its future. This tagline exists for many generations taking a 
new form from one era after another. The entire timeline notes three major 
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education themes that directly influence the society: Education 1.0 
underscored agriculture education for agriculture industry, Education 2.0 
responded to industrial society, and Education 3.0 addressed the technology 
society (Diwan 2017). Currently, the quest for industrial revolution 4.0 (IR 
4.0), dominated by global connectivity, smart machines and new media, 
dictates the new contour of education to foresee training and re- and 
upskilling of the future workforce of this era (Haron 2018). Education in this 
digital era known as Education 4.0 (E 4.0) and labeled as the “Dawn of Digital 
Monarchy” envisions to facilitate educating the Generation Z (13-19 years old) 
learners to imbibe life skills and skills of creating innovation to develop the 
future workforce 4.0 who exhibits the 4lIs: intelligent, interconnected, 
instrumented, and innovative (Goldsberry 2018) with a “learning is a way of 
life’ mindset (Shook and Knickrehm 2017). This education framework 
emphasizes a new vision for learning where content and subject matter are 
secondary to the knowledge of why you need something, and where to find it 
(Fisk 2017). It features learning together and peer learning where teachers 
primarily act as facilitators of the learning mechanism dictated by the learners, 
and machines aid the facilitators in tracking the learners’ performance 
through data-based customization. This schema especially applies to STEM 
education, which later was known as STEAM (Ghanbari 2015) or Science, 
Technology, Engineering, Agri/Fisheries and Mathematics education in an 
effort to integrate design and creativity, - one of the most sought-after main 
pillars of knowledge-based society and economy (Ministry of Finance of the 
Slovak Republic 2018). While E 4.0 frames the new learning paradigm for 
workforce 4.0, the new education framework calls for quality, re and upskilled 
teachers to catalyze learning in IR 4.0. 


Quality and re and upskilled teachers direct STEM/STEAM education 
to quality, which can prepare the future workforce with skills that match the 
new skill demands of IR 4.0 (Deloitte 2015). Furthermore, they need to 
facilitate STEAM learning for the Generation Z learners to acquire a strong 
background on the meta-discipline (Morrison 2006; Tsupros et al. 2009; 
Ejiwale 2013), and to obtain specific and highly intricate skills such as design 
thinking, time management and programming skills (Montero and Evans 2011; 
Mars et al. 2016) aside from life skills and the 4Is. STEAM educators 
perpetuate learning as a way of life for the Generation Z learners to survive 
and be successful in their future work environment (Renjen 2018). 


The demand for quality STEAM education to be in the loop of 
Education 4.0 framework forges the need for re and upskilled competencies 
of STEAM teacher quality. As Wilson (2016) claimed, teaching in this era 
should go beyond the teaching of disciplinary subject matter, instead, teachers 
should focus on the integrative presentation of lesson to learners, e.g. 
authentic/problem-based learning and design thinking, emphasizing 
interdisciplinary approach using STEAM field to teach a STEAM course. 
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Feedback system and assessment should emphasize formative development of 
students’ skills to demonstrate their learning and its relevance to society 
(Miller 2017). Finally, quality STEAM teachers should showcase dynamic 
professional development as pathways to success (e.g. 
collaborative/partnerships, community service for service learning, and 
professional learning). These traits feature the new skills of quality STEAM 
teachers who are driven by what they know and what they can do with what 
they know (Obama 2016) to train our Generation Z learners to learn the trick 
of how to solve real problems and be contributory to the society. 


There are already numerous efforts to provide quality education by 
improving the quality of teachers. Training and upskilling of teachers in other 
countries commenced to attain the aforementioned traits and new skill set 
required to be quality STEAM teachers of the Generation Z learners which 
calls for monitoring and assessment tools. In fact, the National Research 
Council proposed two indicators for STEM teacher quality: teachers’ science 
and content knowledge for teaching and teachers’ participation in STEM- 
specific professional development. Similar efforts surfaced in the STEAM 
world (Kim and Kim 2016) extending the attempt from identifying and 
polishing the new teacher quality competencies to developing STEAM quality 
teaching indicators and rating tools. 


In the Philippines, standards for professional teachers Philippine 
Professional Standards for Teachers (PPST) set the tone of research activities 
related to teacher quality (Department of Education-Teacher Education 
Council 2017). Specifically, PPST outlines the desired competencies and skills 
of quality teachers to enable them to handle and manage emerging global 
frameworks. However, PPST targets the primary, junior, and senior education 
level with no existing elaborations on subject matter or content, and teaching 
and learning of complex skills in the tertiary level, thus, enjoining the group 
of Morales et al. (2019) to design a self-rating proficiency indicator for tertiary 
STEAM (with “A” for Agri/Fisheries) teachers framed from the paradigms of 
PPST; Policies, Standards and Guidelines of the Philippine Commission on 
Higher Education (PCHED); and the theoretical underpinnings of 
Technological Pedagogical Content Knowledge (TPACK) (Mishra and Koehler 
2006). This self-rating tool boosts the disciplinal mapping and matching of 
the seven domains of PPST and the seven TPACK dimensions to capture the 
entire spectrum of competencies expected of a Philippine tertiary STEAM 
quality teacher. Apparently, Kim and Kim (2016) tracked similar path 
specifying constructs and domains to which teacher quality may be gauged 
through a self-rating tool (Kim and Kim 2016). However, minority among the 
identified studies in an exhaustive literature search present the assessment 
framework of the developed self-rating tool, thus the current investigation 
focuses on developing the corresponding assessment framework for the self- 
rating proficiency indicators for Philippine STEAM educators that highlights 
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the scoring system for the self-rating tool. Specifically, the study sought 
answers to the following objectives: 1. Design the mathematical framework of 
scoring STEAM educators’ proficiency; 2. Design the program for scoring 
STEAM educators’ proficiency; 3. Validate the designed assessment/scoring 
program; and 4. Try-out and pilot test STEAM educator’s proficiency scoring 
framework. 


METHODS 


The proficiency scoring framework crafted and elucidated in the 
following is a product in response to a fundamental question that we think the 
formalism should be able to answer: How do we extract and therefore 
determine ones proficiency from the self-rating survey data alone without any 
external assumption and thereby self-contained? The answer to this is 
exemplified in the next paragraphs, including an illustration determining the 
proficiency of a higher education STEAM teacher. A three tier validation, 
involving both quantitative and qualitative validations, were performed to the 
scoring framework and program. The results of pilot testing can be seen in the 
next section. 


We wish to know the national STEAM educators’ proficiency profile in 
a) the seven domains of the PPST, and b) the seven TPACK dimensions, from 
data gathered in the online administration of the developed self-rating survey 
(Morales et al. 2019) to n higher education STEAM teachers nationwide. The 
PPST’s seven domains are in 1) content knowledge and pedagogy, 2) learning 
environment, 3) diversity of learners, 4) curriculum and planning, 5) 
assessment and reporting, 6) community linkages and _ professional 
engagement, and 7) personal growth and professional development. The seven 
TPACK dimensions are 1) pedagogical content knowledge (PCK), 2) 
technological pedagogical knowledge (TPK), 3) technological pedagogical 
content knowledge (TPCK), 4) technological content knowledge (TCK), 5) 
technological knowledge (TK), 6) pedagogical knowledge (PK), and 7) content 
knowledge (CK). We express the STEAM educators’ proficiency as a) beginner, 
b) proficient, c) highly proficient, and d) distinguished in each of the seven 
domains and seven dimensions. We include as well the proficiency profile and 
proficiency in the overall domain. 


National Proficiency Profile 


Let Rij,, the STEAM educator’s proficiency profile of the j‘” higher 
education STEAM teacher-respondent in the k*" PPST domain, be defined as 
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total i‘ choice j*" respondent chose in items in the k*” domain 
ijk Oe he (1) 
total items in the k** domain 

where i = 0,1, 2,3, 4, with i = 0 corresponding to choice “Not Applicable” 
(NA), i = 1 corresponding to choice “Rarely true to myself” (Rarely-ttm), i = 
2 corresponding to choice “Occasionally true to myself” (Occasionally-ttm), 
i= 3 corresponding to “Often true to myself’ (Often-ttm), and i=4 
corresponding to “Always true to myself” (Always-ttm), as choices on all items 
of the self-rating survey; j = 1, 2,...,n; and k = 1,2,...,7. Similarly, we define 
the STEAM educator’s proficiency profile of the j*” higher education STEAM 
teacher-respondent in the k‘" TPACK dimension, 7; ;, as 


total i” choice j*” respondent chose in items in the k“ dimension (2) 


ijk = total items in the k‘” dimension 


Ri jx and T;;, obey the normalization property 


4 4 
>. Rit = > Tit = 1, = 1, (3) 


because each teacher-respondent is required to respond on every item by 
choosing one and only one choice. 


The national STEAM proficiency profile of the sample population 


corresponding to the i*” choice in the k‘” domain and k‘" dimension are 
measured with the following means given by 


1 n 1 n 
Rix ==) Rip, T ix =-) Ti jx: (4) 
j=l y=1 


Each domain contributes equally to the overall proficiency profile of the 
sample population. The domain-based overall proficiency profile then is 


Gi = 3 Rix: (5) 


The national STEAM educators’ proficiency profiles in eq. (4) and eq. (5) 
satisfy the normalization conditions 
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Sie Sankt ma YG =1 (6) 


which follows directly from eq. (3). 
Knowing Ones Proficiency Profile and Proficiency 


Any STEAM teacher wishing to check his/her level of proficiency may 
do so by taking the 60 item self-rating survey. Once completed, the teacher’s 
proficiency profiles are calculated using 


ris n 
Rix = Rite = >: Or; Rijx, Tix = Tix = >: 61; Tijx- (7) 
j=l j=l 


0 ' FF s opRAM 
1 l=j 


teacher’s proficiency profile also satisfy the normalization conditions 


Sta= Ste =he=t (8) 


ASTEAM teacher’s proficiency profile isthen compared to the national 
STEAM educators’ proficiency profile, calculated using eq. (4), by taking their 
difference like so AR, = Rix — Rix and AT; = Tix — Tix . The proficiency 
profiles’ normalization properties in eq. (3), (6), and (8) will ensure that, for 
any PPST domain or TPACK dimension, any one and only one of the five 
differences AR4,, AR3x%, AR2x, AR;,, and ARox (AT4;, AT3x, AT2x, ATi, and ATox) 
will obtain the greatest positive difference, thereby determining uniquely the 
teacher’s proficiency using the difference-proficiency association in Table 1. 
Print out or email generated by the program spells out the general attributes 
and the per domain attributes of the teacher’s proficiency level. 


where 67; is the Kronecker delta having the property 6); = 


Table 1. Translating difference in proficiency profile to teacher’s proficiency. 


pat POStuNE STEAM Teacher’s Proficiency 
Difference 
AR 4x (ATy x) Distinguished (i = 4) in domain (dimension) k 
AR3;(AT3x) Highly Proficient (i = 3) in domain (dimension) k 
AR (AT> ) Proficient (i = 2) in domain (dimension) k 
AR, (ATi; ) Beginner (i = 1) in domain (dimension) k 
ARox(ATox) Not Observed (i = 0) in domain (dimension) k 
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In the following illustration, the scoring framework is applied to n = 
1507 higher education STEAM educators who had participated in the online 
self-rating survey at the time this investigation started which is roughly 78% 
of the total sample population of the study provided by 123 randomly selected 
universities and colleges from a total of 2,299 Philippine higher education 
institutions. 


For instance, the self-rating survey data says that the national STEAM 
educators’ proficiency profile of higher education institutions corresponding 
to PPST’s domain on content knowledge and pedagogy, calculated using eq. 
(4) with k = 1,n = 1507, are as follows explicitly: Always-ttm (i = 4), ae = 
0.37; Often-ttm (i = 3), R,, = 0.44; Occasionally-ttm (i= 2), R,, = 0.14, 
Rarely-ttm (i = 1), R,, = 0.03, NA (i= 0), R,, = 0.02 . Suppose now, a 
higher education STEAM teacher who has taken the 60-item self-rating survey 
have a proficiency profile in the PPST’s content knowledge and pedagogy 
domain given explicitly by: Always-ttm (i = 4),R4, = 4/19 = 0.21, Often-ttm 
(i = 3),R3, = 6/19 = 0.32 , Occasionally-ttm (i = 2),Ro, = 5/19 = 0.26 , 
Rarely-ttm (i = 1),R,, = 3/19 = 0.16, NA (i = 0), Ro; = 1/19 = 0.05. Using 
Table 1, we say that the teacher is a beginner STEAM teacher in the PPST’s 
content knowledge and pedagogy domain. The illustration just shown is 
extended to the other PPST domains, TPACK dimensions and overall PPST 
domain to complete the level of proficiency unique to the teacher. 


Scoring Program and Validation 


The derived mathematical eq. (4) and eq. (5) directed the development 
of the scoring programs using Microsoft excel and Fortran. Three tier 
validation (quantitative and qualitative) through participant responses 
determined the robustness and soundness of the scoring program. For the 
quantitative validation, the sampling ensured nationwide coverage. 
Respondents received the survey and replied to them online using google form. 
Once all prospective replies in google form are in, these are converted into 
Excel file for the convenient and automatic calculation of the a) ratios in eq. 
(1) and (2); and b) means in eq. (4) and (5). Likewise, the proficiency profiles 
in eq. (4) and (5) are calculated independently using Fortran. The Fortran 
codes are produced and saved as f95 file with respondents’ replies converted 
into input txt file. These files are compiled to produce the proficiency profiles. 
Comparison and equivalence of the proficiency profiles established through 
the scoring programs using Microsoft Excel and Fortran determined the first- 
tier quantitative validation of the programs. 


The second tier validation involves the calculation and comparison of 
the proficiencies of STEAM teachers, who 1) have taken the online, self-rating 
survey, and 2) were observed in classrooms. The proficiencies were 
determined through the Fortran implementation of the scoring framework 
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using data from a) the self-rating survey, and b) the classroom observation 
ratings of observers. Validity is established once the program results show the 
presence of agreement on the proficiencies of teachers obtained a) according 
to oneself, and b) from an observer in the classroom. 


The third tier emphasized a qualitative validation by comparing 
generated codes in the interview transcript and observation notes of the 
participant in each career stage emerging as incurring the same measure in 
the online survey and in the classroom rating scale (2"‘ tier) and the significant 
attributes underscored in all PPST domains and strands in each domain and 
TPACK dimensions. The proponents noted and assessed the equivalence of 
the indicators in the online survey as clustered in each domain for equivalence 
and presence in the interview transcript and observation notes to establish the 
validity. 


RESULTS 


The implementation of the analytic expressions of the proficiency 
profiles, including the determination of proficiencies of teachers, specified in 
the methods section are herein explained in detail. The results of validations 
and pilot tests are presented here as well. 


Figure 1 shows the result (the value appearing in element T597) of the 
Excel implementation of eq. (1) corresponding to a teacher-respondent’s 
G = 597 — 1) “always true to myself” choice (i = 4) on the 19 items of Domain 
1 (k = 1). The ratios appearing in columns U, V, W, and X, were for “often true 
to myself” (i = 3), “occasionally true to myself” (i = 2), “rarely true to myself” 
(i = 1), and “not applicable” (i = 0) choices, respectively. In each and every 
row, the elements in columns T, U, V, W, and X, sums up to unity, consistent 
with the normalization property, eq. (3), of the proficiency profile. The same 
Excel implementation of eq. (1) was done to the other 6 domains. The same 
Excel implementation was employed to the 7 dimensions using eq. (2). 


Figure 2 shows the results of the Excel implementation of eq. (5). 
Elements BI1533, BJ1533, BK1533, BL1533, and BM1533, corresponding to 


G4, G3,G>, Gy, and Gy in eq. (5), respectively, are collectively called the overall 
national STEAM proficiency profile in the PPST domain. Element BN1533 
confirms the normalization property of the G;'s in eq. (6). The numbers 
appearing in elements BI1512, BJ1512, BK1512, BL1512, BM1512 are results of 
Excel implementation of R41, R31, R21, R11, and Ro; in eq. (4), respectively. 
These elements collectively correspond to the national STEAM proficiency 
profile in the PPST’s content knowledge and pedagogy domain. Elements 
BN1512, BN1515, BN1518, BN1521, BN1524, BN1527, and BN1530 verify that 
the R;;,'s satisfy the normalization property eq. (6). 
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Figure 3 shows the initial part of the Fortran program code named 
proficiency that implemented eq. (1), eq. (4) and eq. (5). It starts by uploading 
(using the Fortran command open(10,file="Domain1.txt’)) the responses of 
the 1,507 teacher-respondents to the 19 items covering domain one stored in 
a data file named Domain1.tex. Henceforth, until after the 4 “end do” Fortran 
statement, eq. (1) is implemented, for each response of every teacher- 
respondent in domain 1, and stored in codes R4(j), R3G), R2(Gj), RiG), and 
Ro@). 


program proficiency 


integer, dimension(2e00,20) :: f,ff 
real, dimension(10,13) :: NAF1 

real, dimension(2000) :: R4 

real, dimension(2000) :: R3 

real, dimension(200@) :: R2 

real, dimension(200@) :: R1 

real, dimension(200@) :: R@ 

integer ssi 

integer HA: | 

real :: answer,x,y,x1,y1,z1 


real :: DIF10,HDF10,PRF10,B8IF10,NAF10 
real :: DIF11,HDF11,PRF11,B81F11,NAF11 
real :: NORMAL,NORMAL1,NORMAL2 
real :: RR4,AVR4,AVR41,AVR42 
real :: RR3,AVR3,AVR31,AVR32 
real :: RR2,AVR2,AVR21,AVR22 
real :: RR1,AVR1,AVR11,AVR12 
real :: RR@,AVRO,AVRO1,AVRO2 
real :: NORMAL3,NORMAL4,NORMALS 
real :: AVR43,AVR44,AVR45 
real :: AVR33,AVR34,AVR35 
real :: AVR23,AVR24,AVR25 
real :: AVR13,AVR14,AVR15 
real :: AVR@3,AVR@4,AVR@5 
real :: NORMAL6,NORMAL7 
real :: AVR46,AVR47 
real :: AVR36,AVR37 
real :: AVR26,AVR27 
real :: AVR16,AVR17 
real :: AVR@6,AVRO7 
open(1@, file='Domaini.txt') 
do j=1,1507 
do i=1,1 
read(10,*) f(j,i),f(j,i+1),f(j,i+2),f(j,1+3),f(7,i+4), f(5,1+5), f(5,i+6), f(5,1+7), f(5,1+8), f(5,1+9), f(j,i+10), f(5,i+11) & 
of(j,i+12), 05,1413), f(5,i+14), €(5,i+15), f(5,1+16), f(5,i+17), (5, i+18) 
end do 
end do 
do j=1,1507 
DIF11=8 
HPF11=@ 
PRF11=@ 
BIF11=@ 
NAF11=@ 
DIF10=¢@ 
HPF10=6 
PRF10=@ 
BIF10=0 
NAF10=0@ 
do i=1,19 
if (f(j,i)==4) then 
DIF11=DIF10+1.0 
else if (f(j,i)==3) then 
HPF11=HPF10+1.0 
else if (f(j,i)==2) then 
PRF11=PRF16+1.0 
else if (f(j,i)==1) then 
BIF11=BIF10+1.0 
else if (f(j,i)==5) then 
NAF11=NAF10+1.0 
else 
print *, f(j,i) 


Figure 3. Fortran implementation of equation 1. 
Then, from the last “do — end do” loop statement until the start of the 


implementation of domain 2, eq. (4) is implemented for domain 1, stored in 
codes AVR41, AVR31, AVR21, AVR11, AVRO1, and printed, as shown in Figure 


4. 
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HPF10=0 
PRF10=0 
BIF10=0 
NAF10=@ 
do i=1,19 
if (f(j,i)==4) then 
DIF11=DIF10+1.0 
else if (f(j,i)==3) then 
HPF11=HPF10+1.@ 
else if (f(j,i)==2) then 
PRF11=PRF10+1.0 
else if (f(j,i)==1) then 
BIF11=BIF10+1.0 
else if (f(j,i)==5) then 
NAF11=NAF10+1.@ 
else 
print *, f(j,i) 
end if 
DIF1@=DIF11 
HPF1@=HPF11 
PRF10=PRF11 
BIF1@=BIF11 
NAF1@=NAF11 
R4(j)=DIF11/i 
R3(j)=HPF11/i 
R2(5)=PRFAI1/i 
R1(5)=BIFA1/i 
RO(5)=NAF11/i 
NORMAL=R4 (5 )+R3(j )+R2(5)+R1(5)+RO(5) 
end do 


do j=1,1507 
RR4=RRG+RG( 5) 
RR3=RR3+R3(5) 
RR2=RR2+R2(j) 
RR1=RR1+R1(j) 
RR@=RRO+RO(j) 
end do 
AVR4=RR4/ (5-1) 
AVR3=RR3/(j-1) 
AVR2=RR2/(j-1) 
AVR1=RR1/(j-1) 
AVRO=RRO/(j-1) 
NORMAL=AVR4+AVR3+AVR2+AVR1+AVRO 
AVR41=AVR4 
AVR31=AVR3 
AVR21=AVR2 
AVR11=AVR1 
AVR@1=AVRO 
NORMAL1=NORMAL. 
print *, AVR41, j-1 
print *, AVR31, j-1 
print *, AVR21, j-1 
print *, AVR11, j-1 
print *, AVR@1, j-1 
print *, NORMAL1 
{Start of Domain 2 


Figure 4. Continuation of Figure 3 including the Fortran implementation of 
eq. (4) 


The procedure just described is repeated for the remaining 6 domains 
in preparation for the implementation of eq. (5) shown in Figure 5. The 
Fortran code program proficiency is then compiled to run the program and 
extract the results into a default file a.out, the result of which is shown in 
Figure 6 and the basis of Table 2. 


The first tier validation says that the national STEAM proficiency 
profile for both PPST domains and TPACK dimensions determined through 
Microsoft excel agree with Fortran results with 1,507 as the total number of 
sample respondents. 
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DIF10=DIF11 
HPF1@=HPFI2 
PRF1@=PRF1 
BIF10=BIF11 
NAFI@=NAFA2 
R4(j)=DIFAI/i 
R3(j)=HPFII/i 
R2(j)=PRFLI/i 
Ri(j)=BIFII/i 
RO(j)=NAFII/i 
NORMAL=R4 (5 )+R3(j)+R2(5)+R1(j)+RO(5) 
end do 
end do 
RRG=O 
RR3=@ 
RR2=@ 
RR1=@ 
RRO=B 
do j=1,1507 
RRG=RR4+RG( 5) 
RR3=RR3+R3( Jj) 
RRZ=RR2+R2(j) 
RR1=RR1+R1 (5) 
RRO=RRO+RO( 5) 
end do 
AVR4=RRG/(j~1) 
AVR3=RR3/(j-1) 
AVR2=RR2/ (5-1) 
AVR1=RR1/(j-1) 
AVRO=RRO/ (5-1) 
NORMAL=AVR&+AVRS+AVR2+AVR1+AVRO 
AVRG7=AVRG 
AVR37=AVR3 
AVR27=AVR2 
AVRI7=AVRI 
AVRO7=AVRO 
NORMAL7=NORMAL 
print *, AVR47, 
print *, 
print *, 
print *, 
print *, 


AVR4=(AVRG1+AVR42+AVR43+AVRG4+AVRGS+AVRGG+AVRG7) /7 
AVR3= ( AVR31+AVR32+AVR33+AVR34+AVR3I5+AVRI6+AVR37) /7 
AVR2=(AVR21+AVR22+AVR23+AVR24+AVR25+AVR26+AVR27 ) /7 
AVR1=(AVR11+AVR12+AVRI3+AVR14+AVRIS+AVRIG+AVRI7)/7 
AVRO=(AVRE1+AVRO2+AVRO3+AVRO4+AVROS+AVROG+AVRE? ) /7 
NORMAL=AVR4+AVR3+AVR26AVR2. 

print *, AVR4 

print *, AVR3 

print *, AVR2 

print *, AVR1 

print *, AVR@ 

print *, NORMAL 

end program proficiency 


(END) 
Figure 5. Fortran implementation of equation 5. 


login: Fri Dec 7 on tty: 
tpack-steam$ cd Proficiency/ 

roficiency tpack-steam$ gfortran OverallDomain. 95 
Proficiency tpack-steam$ ./a.out 


1507 


2.12143420E-@2 1507 
1.31386947E-@2 1507 
2.00000036 
0.586974688 1507 
0.34372887¢ 1507 
5.36544099E-82 1507 
8.15243468E-03 1507 
1507 
1807 
1807 
7.69005343E-02 1507 
1.22391675€- 1507 
9. 953544486. 1507 
1.e0000036 
©. 453882277 1507 
0.425348103 1507 
8.58217999E. 1507 
1.50409220E 1507 
1.99071132E-@2 1507 
2.00000026 
© ,628012097 1507 
0. 358998402 1507 
7. 91545659 1507 
1.63048655E-@2 1507 
1.75372066E-@2 1507 
0.999999046 
505507231 1507 
0.375681056 1507 
8.20171237E-02 1507 
2,.09688265E-02 1507 
1.59256775E-@2 1507 


0.999999948 


1.00000012 
TPACK-STEAMS~-MacBook-Pro-3:Proficiency tpack-steams ff 


Figure 6. Compilation of Fortran code with gfortran and viewing of results 
with ./. 
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Table 2 shows the national STEAM proficiency profile based on the 
PPST domains. On the average, it appears that, overall nationally, for a “rarely 
true to oneself’? choice chosen on an item, five other items received 
“occasionally true to oneself” response, while “often true to oneself” choice 
were chosen in 24 other items, and 30 other items got “always true to oneself” 
choice. 


Table 2. National STEAM educator's proficiency profile in the Philippine 
Professional Standards for Teachers (PPST) domains. ** true to myself are 
herein implied for simplicity from here onwards. 


PPST Domain Always™ | Often™ | Occasionally” | Rarely™ | NA 
Content Knowledge 
and Pedagogy 0.37 0.44 0.14 0.03 | 0.02 
Learning 0.52 0.39 0.07 0.01 | 0.01 
Environment . ; ; . ; 
Diversity of Learners 0.59 0.34 0.05 0.01 | 0.01 
Curriculum and 
Planning 0.50 0.40 0.08 0.01 | 0.01 
Assessment and 
Reporting 0.45 0.42 0.09 0.02 | 0.02 
Community Linkages 
and Professional 0.53 0.36 0.08 0.02 | 0.01 
Engagement 
Personal Growth and 
Professional 0.50 0.38 0.08 0.02 | 0.02 
Development 
Overall 0.49 0.39 0.08 0.02 | 0.02 


Table 3 shows the national STEAM educator’s proficiency profile based 
on the 7 TPACK dimensions. On the average, out of the 13 items of the self- 
rating survey dealing directly on the pedagogical knowledge dimension, 
around seven items received “always true to myself’ response, roughly five 
other items got “often true to myself” answer, and close to an item obtain an 
“occasionally true to myself” reply. Choices “rarely true to myself’ and “not 
applicable” were hardly chosen in this dimension. 


For the second-tier validation, the online, self-rating survey’s sample 
population used was 1,455 teacher-respondents, the proficiency profiles of 
which (see Table 4) in domains 1) content knowledge and pedagogy, 2) 
learning environment, and 3) diversity of learners, formed the basis for 
determining the proficiency level of 52 other teachers who took the online 
survey and were observed in classroom as well. 
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Table 3. National STEAM educator's proficiency profile in the Technological- 
Pedagogical-Content-Knowledge (TPACK) dimensions. 


TPACK Dimension Always | Often | Occasionally | Rarely | NA 
Pedagogical Content 
Knowledge 0.54] 0.36 0.07 0.01 | 0.02 
Technological 
Pedagogical Knowledge 0.50 0.41 0.07} 0.01] 0.01 
Technological 
Pedagogical Content 0.30 0.44 0.18 0.05 | 0.03 
Knowledge 
ae ont 0.43] 0.43 0.10 | 0.02] 0.02 
Sareea 0.48) 0.38 0.09} 0.02} 0.03 
Pedagogical Knowledge 0.56 | 0.37 0.06 | 0.01 | 0.00 
Content Knowledge 0.45| 0.44 0.09 0.01] 0.01 


Table 4. STEAM proficiency profile of 1455 online survey teacher-respondents. 


Domain Always | Often | Occasionally | Rarely | NA 
Content Knowledge and 
Pedagozy 0.38 0.43 0.14 0.03 | 0.02 
Learning Environment 0.52 0.39 0.07 0.01 | 0.01 
Diversity of Learners 0.59 0.34 0.05 0.01 | 0.01 


Finally, Table 5 presents the third tier of validation focused on 
identifying presence or occurrence of indicators in the interview transcripts 
and classroom observation notes matched with the expected attribute per 
career stage as per PPST. 


Table 6 shows the STEAM proficiency profiles of a teacher classified as 
distinguished overall (Teacher A), including in all domains except in Domain 
5, where the teacher appears to be highly proficient only; a teacher envisaged 
as highly proficient overall, including in all domains except in Domain 7, 
where the teacher appears to be distinguished (Teacher B); and a teacher 
proficient in Domain 3 and beginner in Domain 1 (Teacher C). 
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Table 5. Qualitative validation of the scoring program. The shaded portion 
indicate non-match of interview transcript or classroom observation notes 
with the expected attribute per career stage as per Philippine Professional 
Standards for Teachers (PPST). Data element having two values mean data 
comes from two respondents. 


Danan Distinguished ree t Proficient | Beginner 
I CON | I | CON, I | CON | I | CON 
Content Knowledge and 
Pedagogy 
e composed of 7 strands 18 144 |11] 10 |10] 10 
e with 19 items (Online 6 
Surve 
Learning Environment 
e composed of 6 strands rs 6 
e with 10 items (Online 3 3 5 4 
Surve 
Diversity of Learners 
e composed of 5 strands 7 ; 6 
e with 7 items (Online 3 : ‘ 5 3 
Surve 
Curriculum and Planning 
e composed of 5 strands 
e with 9 items (Online 3 3 3 a 5 5 ” 7 
Sine 4 3 4 5 
Assessment and Reporting 
e composed of 5 strands r 3 Fe 3 i : A 
e with 3 items (Online 3 1 1 1 
Surve 
Community Linkages and 
Professional Engagement 
e composed of 4 strands 5 1 5 O 3 O 4 2 
e with 7 items (Online 
Survey) 
Professional Growth and 
Professional Development 
e composed of 5 strands 3 2 1 1 1 3 1 
e with 5 items (Online 
Survey) 


I-Interview, CON — classroom observation notes 
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DISCUSSION 


A self-contained mathematical framework, free from external 
assumptions, able to unambiguously determine a higher education STEAM 
teacher’s proficiency is formulated in this study. On the basis of this 
framework, a program is designed and subjected to quantitative and 
qualitative validation showing meaningful and consistent prediction of 
STEAM teachers’ proficiency. A pilot test of this program to three STEAM 
teachers of varying levels of proficiency indeed shows its capacity to identify 
uniquely a STEAM teacher’s proficiency. These results are necessary to make 
sense of the developed self-rating tool in Morales et al. (2019) in providing 
STEAM teachers with their equivalent rating in terms of STEAM Education 
proficiency. Consequently, the STEAM educator’s proficiency scoring 
framework ably determines the proficiency profile of a target population and 
predict unambiguously individual teacher-respondent’s proficiency level. The 
outputs of the previous work and the current study form part of a bigger 
project on developing the Philippine STEAM Education Model for higher and 
advanced learning. The novelty of this study (a) is that it provided the self- 
rating tool in Morales et al. (2019) a programmed framework of scoring which 
other and most developed instruments lack, and (b) lies in the quantitative 
and qualitative combination of approaches to validation of the STEAM 
educators’ scoring framework, which, to the best of our knowledge, is unique 
to this study. 


The developed framework, including the program it took form, in 
determining a Philippine higher education STEAM teacher’s proficiency is 
self-contained, universal, albeit shaped by local peculiarities, transparent, and 
technology-enhanced (Steel 2015). The determination of ones’ proficiency 
depends entirely and sufficiently on the population of the self-rating survey 
through the national STEAM educators’ proficiency profiles as shown in 
Tables 2 and 3. External assumption is not necessary. The scoring framework’s 
adaptability is universal provided the survey used require one and only one 
response on every item, which may be adapted to suit any local setting—a trait 
matching the characteristics of the seven principles of universal design 
(Center for Excellence in Universal Design 2019). The formulated framework 
and program give results that any interested investigator may verify 
independently given the same set of data. 


The framework’s and program’s unique determination of ones’ 
proficiency have been quantitatively and qualitatively validated with varying 
presence of agreement. The presence of agreement in the proficiency level of 
teachers according to oneself and from an observer in the classroom is shown 
in Table 7. Out of the 52 teachers who took the online survey and were likewise 
observed in class, three teachers were determined as proficient in domain on 
diversity of learners in both online survey and classroom observation, which 
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translates to 50% agreement, considering there were six teachers predicted as 
proficient in the domain on diversity of learners through the online survey. 
Except in the domains on learning environment and on diversity of learners, 
both of which practically have no beginners, distinguished teachers in the 
domain on learning environment exhibits the least amount of agreement with 
7%. 


Table 7. Extent of agreement on proficiency level of teachers according to 
oneself and from an observer in the classroom. 


Highly 
Proficient 
Content Knowledge and 1 [11] 9% 3 [20] 15% | 5 [15] 33% | 2 [6] 33% 
Pedagogy 
Learning Environment 1 [14] 7% 8 [32] 25% | 1[5] 20% o [1] 0% 
Diversity of Learners 6 [19] 32% 7 [26] 27% | 3[6]50% o [1] 0% 


Domain Distinguished Proficient | Beginner 


The occurrence of indicators may mean that outputs generated by the 
scoring program match the indicators as perceived by classroom observers 
and interviewers of the participants identified for validation test (Table 5). The 
decreasing trend may imply that in most of the domains, 4 of 7, proficiency in 
STEAM teaching may be dictated by the number of indicators exhibited by the 
STEAM educator(s) in the following domains: domain 1-content knowledge 
and pedagogy, domain 5-assessment and reporting, domain 6-community 
linkages and professional engagement, and domain 7-professional growth and 
development, with the greatest number of combination for the distinguished 
career stage and the least for the beginner stage. For example, distinguished 
teacher in domain 1 as rated by the scoring program based on the online self- 
rating survey exhibited 18 out of 19 indicators as per interview transcripts and 
3 out of 19 as per classroom observation notes with a decreasing combination 
until the beginner stage. However, three of the seven domains did not exhibit 
the same trend, noting greater number of indicators exhibited by other career 
stages compared to the distinguished career stage. Analysis of the sample 
interview transcripts and classroom observation notes in Table 8 shows that 
although the three domains (Learning Environment, Diversity of Learners, 
and Curriculum and Planning) manifested a different trend in terms of 
number of exhibited indicators, the minimally exhibited indicator encompass 
large number of indicators with complex attribute. 
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The distinguished teacher’s conduct of STEAM research and 
utilization of such research in STEAM teaching traverse wide array of domains 
that includes all the three aforementioned domains (Hazelkorn et al. 2015). 
Comparing this exhibited attribute, teachers in other career stages specified 
minute attributes compared to those exhibited by the distinguished teacher, 
indicative of validation of the scoring system developed for the online survey 
to generate the STEAM educator proficiency. 


The scope of the scoring framework developed is universal and may be 
adapted to suit any local setting. Although all necessary aspects of validation 
were done and exhibited favorable results, increasing the number of 
interviews and classroom observations to 10% of the sample population of 
teachers will produce a robust scoring program. 
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